Tools LinkedIn Pricing API FAQ
Login

Frequently asked questions

Keyword SERP scraping

Keywords and search operators

An introduction to keywords and search operators


Each of our jobs may contain up to 50.000 keywords, however it is recommended to use multiple smaller sized jobs instead.
A keyword may contain up to 250 characters, it's recommended to use shorter keywords if possible.


Especially when scraping large amounts of data from Google the keyword length and complexity does matter.
Google prevents scraping through a self-learning system, the more natural a keyword looks the better.

Our service is highly optimized in avoiding scraping detection, however it's good practice to use an organic type of keyword when possible.

The more organic (natural) your keywords look, the less likely Google can detect bulk scraping activity.

The large majority of our clients is not affected by this, however when scraping very specific data from Google this can cause an issue.

Our system will signal and take counter measures which usually still lead to success. 

Please just contact our customer support if you have detailed questions.


When the detection system is triggered, keywords of similar structure might be rejected, in some cases we have seen temporary world-wide bans.
Especially when using "intitle:", "site:" and similar special search operators keywords can be detected by Google.

 

In many cases it's possible to reduce or avoid search operators:

Instead of searching for "best shoes site:ebay.com" you might just search for "ebay.com best shoes"

Searching for ten thousands of shoes using "site:ebay.com" is likely going to result in some skipped keywords from detection.

Searching without "site:" is likely not going to skip any keywords.

 

 


Bing is often a good alternative for pure result harvesting, similar search operators can be used with Bing just as with Google.

Google search operators: https://support.google.com/websearch/answer/2466433?hl=en
Bing search operators: https://msdn.microsoft.com/en-us/library/ff795662.aspx